An Uncommon Task: Participatory Design in Legal AI

Despite growing calls for participation in AI design, there are to date few empirical studies of what these processes look like and how they can be structured meaningful engagement with domain experts. In this paper, we examine a notable yet understudied design process the legal that took place over decade ago, impact which still informs automation efforts today. Specifically, evaluation activities from 2006 2011 within TeXT Retrieval Conference's (TREC) Legal Track, computational research venue hosted by National Institute Standards Technologies. The Track TREC is history practice because it relied on range participatory approaches facilitate new techniques--in case, automating attorney document review civil litigation matters. Drawing archival interviews coordinators TREC, our analysis reveals an interactive simulation methodology allowed computer scientists lawyers become co-designers helped bridge chasm between real-world, high-stakes practice. analyzing case recent past, aim empirically ground contemporary critiques development greater as means address them.